Learning Bayesian Networks under Equivalence Constraints (Abstract)

Machine learning tasks typically assume that the examples of a given dataset are independent and identically distributed (i.i.d.). Yet, there are many domains and applications where this assumption does not strictly hold. Further, there may be additional information available that ties together the examples of a dataset, which we could exploit to learn more accurate models. For example, there are clustering tasks in the domain of semi-supervised learning where, for example, we have available side information that tells us that certain pairs of examples belong to the same cluster. To incorporate such information, constrained versions of k-means clustering (Wagstaff et al. 2001), Gaussian mixture models (Lu and Leen 2004; Shental et al. 2003) and a variety of other models and algorithms, have been proposed in the literature; see, e.g., the surveys (Davidson 2009; Han, Kamber, and Pei 2011). We propose here to abstract such problems in more general terms, as a task of learning from datasets that are subject to equivalence constraints. We formalize the notion of learning a Bayesian network subject to equivalence constraints, introducing a notion of a constrained dataset, which implies a corresponding constrained log likelihood. The constrained log likelihood provides a simple and principled way to learn, for example, the parameters of a Bayesian network from a constrained dataset. The constrained log likelihood, however, is intractable in general, although we identify a special case where we can design practical algorithms for optimizing the constrained log likelihood. In particular, we propose, as an example, a constrained generalization of expectation maximization (EM), for a class of models that subsumes those for constrained clustering tasks as a special case.